GitHub

您所在的位置：网站首页 › 200800485 symmetrynet learning to predict › GitHub

GitHub

2023-08-08 12:26| 来源: 网络整理| 查看: 265

NeRD: Neural 3D Reflection Symmetry Detector

This repository contains the official PyTorch implementation of the paper: Yichao Zhou, Shichen Liu, Yi Ma. "NeRD: Neural 3D Reflection Symmetry Detector". CVPR 2021.

Introduction

We present NeRD, a Neural 3D Reflection Symmetry Detector, which combines the strength of learning-based recognition and geometry-based reconstruction to accurately recover the normal direction of objects' mirror planes. NeRD uses coarse-to-fine strategy to enumerate the symmetry planes and then find the best ones by building 3D cost volumes to examine the intra-image pixel correspondence from the symmetry.

Qualitative Measures ResNet Regression NeRD ResNet Regression NeRD

Errors of the mirror plane are marked in red.

Code Structure

Below is a quick overview of the function of key files.

########################### Data ########################### data/ shapenet-r2n2/ # default folder for the shapenet dataset new_pix3d/ # default folder for the pix3d dataset logs/ # default folder for storing the output during training ########################### Code ########################### config/ # neural network hyper-parameters and configurations shapenet.yaml # example config for shapenet pix3d.yaml # example config for pix3d misc/ # misc scripts that are not important find-radius.py # script for generating figure grids sym/ # sym module so you can "import sym" in other scripts models/ # neural network architectures symmetry_net.py # wrapper for loss mvsnet.py # 3D hourglass config.py # global variables for configuration datasets.py # reading the training data trainer.py # general trainer train.py # script for training the neural network eval.py # script for evaluating a dataset from a checkpoint plot-angle.py # script for ploting angle error curves plot-depth.py # script for ploting depth error curves Reproducing NeRD Installation

For the ease of reproducibility, you are suggested to install miniconda before following executing the following commands.

git clone https://github.com/zhou13/nerd cd nerd conda create -y -n nerd source activate nerd conda install -y pyyaml docopt matplotlib scikit-image opencv tqdm # Replace cudatoolkit=10.2 with your CUDA version: https://pytorch.org/get-started/ conda install -y pytorch cudatoolkit=10.2 -c pytorch mkdir data logs results Downloading the Processed Datasets

Make sure curl is installed on your system and execute

cd data ../misc/gdrive-download.sh 1zdHsSb-xHhY8imIy3uWt_XfBW_YQX4Vh shapenet-r2n2.zip ../misc/gdrive-download.sh 1tzR5WDrbYTFkVu58p8qad6CXw6Bw2yNt new_pix3d.zip unzip *.zip rm *.zip cd ..

If gdrive-download.sh does not work for you, you can download the pre-processed datasets manually for ShapeNet and Pix3D, and proceed accordingly.

Training (Optional)

Execute the following commands to train the neural networks from scratch with four GPUs (specified by -d 0,1,2,3):

python ./train.py -d 0,1,2,3 --identifier baseline config/shapenet.yaml python ./train.py -d 0,1,2,3 --identifier baseline config/pix3d.yaml

The checkpoints and logs will be written to logs/ accordingly.

Pre-Trained Models cd logs/ ../misc/gdrive-download.sh 1VVd11U_8wPtrLKjufvcdbJ59h0AJQwCW 200610-234002-8ee0ad2-shapenet-latest.zip # ShapeNet/Symmetry ../misc/gdrive-download.sh 1HcDCWRbafN4GKC4DzeG17Fo7iXimYI0v 200513-030330-c8e671c-shapenet-finetune.zip # ShapeNet/Depth ../misc/gdrive-download.sh 1Nug2c3u1THbPNmTaiGlR8R508nDHP43Q 201113-224159-ec0e932-pix3d-001008000.zip # Pix3d/Symmetry unzip *.zip rm *.zip cd ..

Alternatively, you can download our reference pre-trained models for ShapeNet (Symmetry), ShapeNet (Depth), and Pix3D.

Evaluation

To evaluate the models with coarse-to-fine inference for symmetry plane prediction and depth map estimation, execute

python eval.py -d 0 --output results/nerd.npz logs//config.yaml logs//checkpoint_latest.pth.tar

The error statistics are printed on the screen and the error metrics are stored in results/nerd.npz. To calculate the error metrics and plot the error-percentage curves, execute

python plot-angle.py python plot-depth.py Acknowledgement

This work is supported by the research grant from Sony, the ONR grant N00014-20-1-2002, and the joint Simons Foundation-NSF DMS grant 2031899. We also thank Li Yi from Google Research for his comments.

Citing NeRD

If you find NeRD useful in your research, please consider citing:

@inproceedings{zhou2021nerd, author = {Zhou, Yichao and Liu, Shichen and Ma, Yi}, title = {{NeRD}: Neural 3D Reflection Symmetry Detector}, year = {2021}, booktitle = {CVPR}, }

【本文地址】

今日新闻

200800485 symmetrynet learning to predict

GitHub

GitHub

今日新闻

推荐新闻